Synthesis of Research on the Effects of Coaching for Aptitude and Admissions Tests

tudents are sometimes ,cry' critical of aptitude and admissions tests, which they say focus on trivia, ask tricky questions, and are not relevant to what is taught in schools. But such criticism is easy to dismiss. Who would expect students to be objective about tests that can limit their educational options? Recently, however, several educational researchers have joined students in criticizing aptitude and admissions tests. Slack and Porter (1980) charged that scores on the Scholastic Aptitude Test, the most widely used of all such tests, can be influenced substantially by coaching programs and that the test is not very good at predicting college grades. Jencks and Crouse (1982) coneluded that tests such as the SAT do not measure what we normally mean by "aptitude" any better than do conventional achievement tests. SAT scores appear to be just as dependent on home environment and school quality as are scores on conventional achievement tests. And SAT scores are not better than achievement test scores in predicting success in college and adult life. These critics of aptitude testing have taken on one of the important rites in American education. Every year millions of students take such tests as the Scholastic Aptitude Test, Graduate Record Examination, and the Law School Admissions Test. And more than 50,000 students each vear enroll in commercial coaching schools to prepare themselves for those tests. Many more students take coaching courses offered in their high schools. One survey estimated that one-third of the private and public schools in the Northeast offered some sort of SAT preparation course the other halnd, stated that the SA'I' and (Aldermia and Powers, 198(i). similar tests are largel! resistant to the Teachers and school administrators effects of drill anid practice. The trustees are naturally interested in knowiig of the College Board, for example, rcmore about the best ways of preparing ported that the average increase to be students for tests. Questiolns loom inp on expected from intensive coaching on the all sides: SAT amounts to less than I standard * Can performance on aptitude tests deviations, or less than ten points oin the be influenced bv coaching programs? SAT score-scale of 200 to 800 (College Are aptitude tests less coachable than Entranlce Examnilationl Board, 1968. p. achievemrnent tests' 4). They therefore siewed coaching pro* low much coaching is optimal? grams as a waste of time and imoncy. * What sorts of programs offer the T'lhere are at least two reasons for best preparation for the tests? these incoisisteicies iii conclihsiolis * In order to ensure equal access to One is that different rcviewcrs lasXe coaching, should schools develop their looked at different parts of the literature own coaching programs ()Or should they ofn coaching. None of the asailable encourage enrollment in commercial reviews has come close to including the courses? I entire range of relevant studies. Allother Reviews of research and evaluation problem is that reviewers have not all findings often investigate these quesanalyzed study results with objective. tions, but the reviewers have not statistical tools 'The use of such obiccreached a consensus on coaching protive methods forces rexiewcrs to put grams. Vernon (1954) summarizing the their notions to a quantitative test. Rcfindings of a number of British experts viewers who use less formal methods in one of the first reviews of thts area, often see what they expect to sec in a reported that the average effect of coachcollection of results. ing and practice was to increase aptitude Recent research syntheses that we scores by nearly .6 standard deviations, completed recently at the University of equivalent to nine points on an IQ Michigan were desigiled to overconie scale. Vernon pointed out that such an these major problems (Bangert-DLrowns, effect could be achieved in a remarkably Kulik, and Kulik, in press, Kulik, Banshort time, usually between three and gert-Drowns, and Kulik. in press; Kulik. nine hours, and recommended that all Kulik, and Bangert-Drowns, in press) students be coached. Later reviews, on The primary goal of our project was to evaluate the effectiveness of coachiing for aptitude tests. But to provide a conRobert L. Bangert-Drowns is Research text for interpreting our major results. Assistant, James A. Kulik is Research we also included in our analysis studies Scientist, and Chen-Lin C. Kulik is of practice on aptitude tests and studies Assistant Research Scientist, Center for of coaching and practice on achievcResearch on Learning and Teaching, ment tests. The pool of studies included The University of Michigan, Ann Arbor. in our analyses was thus unusually